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ABSTRACT 

Institutions ot higher education ore embracing the role of the Internet as a medium to promote on-demand 
communication between faculty and students, As such, online course management systems have become an efficient 
and effective means of facilitating learning outside the classroom. To ensure that a course management system meets the 
needs and goals of an institution, it is vital that the selection and migration of content to an online courseware platform is 
done systematically. This article highlights key stages in the selection and migration process. In addition, practical 
recommendations are provided focusing on sensitivity to an institution's culture, resources, political climate, and goals for 
quality and growth. 


Selecting and Implementing a Course 
Management System 

In the emerging age of higher education, technology is 
rapidly transforming the manner in which information is 
stored, transmitted, and retrieved (Apps, 1994), Institutions 
of higher learning are experiencing a paradigm shift which 
embraces the online learning community that meets the 
needs of diverse and dispersed learners. As such, many 
institutions are utilizing course management systems to 
facilitate online, on-demand communication between 
faculty and students, As technology functions to enable 
student learning in ways not previously possible, "effective 
integration of technology is achieved when students are 
able to select technology tools to help them obtain 
information in a timely manner, analyze and synthesize the 
information, and present it professionally" (Kelly, 2000). With 
the reported growth and increasing demand in e-learning, 
institutions are searching for a systematic approach to 
select online courseware platforms. 

A variety of reasons exist for institutions to migrate to an 
online courseware provider. An institution may either be 
integrating course management technology for the first 


time or may have an operational e-learning solution and 
be searching for a new online courseware platform. 
Typically, the need to select an online course 
management system falls into one of four general 
categories: 

1 .Promotion ofe-Learning - Programs that have not utilized 
a course management system previously may be looking 
for options to allow for the introduction and integration of a 
facilitated educational delivery system, 

2. Program Growth - in some cases, a program's success 
may simply outpace the current course management 
provider's capabilities or features. In these cases, it 
becomes necessary to seek out providers that can meet 
the unique needs of an established program. 

3. Due Diligence Technology Review - A reality with today's 
changing technology is the need to stay updated with 
technologies available, As new technologies become 
available, an institution may need to examine the 
availability of the emergent features within various course 
management systems, 

4.Inaccessible Technology - The technology market is 
rapidly evolving. As a consequence of this growth, course 
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management providers may merge, be purchased by 
another company or may simply dissolve in the 
competitive market. 

The expanded selection of competitive online courseware 
platforms allows a university to become a new type of e- 
learning consumer, in the position to require a variety of 
features and functionality in the online courseware 
platform. Institutions should be systematic in how they 
approach the selection and migration of content to the 
online courseware platform, and this article outlines one 
institutions journey through migrating to a new online 
courseware platform. 

The process of selecting an online course management 
system can be timely and cumbersome, but the 
importance of the task mandates that it be done with 
diligence and planning. Key stages in the selection process 
include: 

1. Conduct a needs analysis, 

2. Develop a request for proposals (RFP), 

3. Evaluate RFP responses, 

4. Conduct campus visits, 

5. Select finalist and complete negotiations, and 

6. Create implementation and user education plan. 

Conduct a Needs Analysis 

The first step in selecting a platform entails conducting a 
needs analysis within the institution to evaluate program 
strengths and weaknesses. The focus at this point is not on 
the isolated capabilities of the course management 
system, rather the unique needs of the university that 
dictate what type of course management features are 
needed for a particular environment or program, When 
completing the needs analysis, one must examine the 


features of the course management system, hosting 
capabilities and service/support. 

The number of course management systems available is 
matched only by the range of different features offered by 
each system, variations in whether the course 
management system is hosted by the service provider or 
by the institution, and options in service and support. One 
of the first steps in preparing to select a course 
management system is to evaluate system features and 
capabilities as mandatory, optional, or unnecessary for the 
needs of the institution. In our analysis, we identified 88 
individual features targeting 6 major areas: course 
features, testing features, grade book, course authoring 
tools, other services, online campus administration. The 
individual features considered are listed in Appendix A, 

During this analysis, it is important to consider the type of 
program (supplement, hybrid, or online), current and 
projected program enrollment, number and type of 
courses offered, instructional technology department 
capabilities, budget, institutional goals and mission, and 
faculty/student experience with online systems. With the 
wide range of potential stakeholders that are impacted by 
the course management system, it is essential to involve all 
parties early in the evaluation process. The collaboration 
between faculty- administration, students, and computer 
services will ensure that the final course management 
product can be effectively implemented while promoting 
maximum educational benefits. 

Develop a Request for Proposals 

The request for proposals should identify the unique and 
specific needs of the institution, highlighting the desired 
courseware features and functionality, in addition to 
administrative program support needs. 

After completing the institutional needs analysis to specify 
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the desired features and functionality, we recommend 
creating a matrix of the specific features, services and 
capabilities for which the potential course management 
systems will be evaluated. Two resources we found helpful 
in creating a comprehensive list/matrix include the Western 
Cooperative for Educational Telecommunications (WCET) 

Edutools (http://www.edutools.info/course/) and the 
Learning Management System Evaluation Framework by 
FredBeshears(http://istsocrates.berkeley,edu/fmb/articles/l 
ms_eval/index,html). 

In reviewing application software providers, it is important to 
also consider the technical support and services the 
institution will need from the courseware. Such items may 
include: 

• Migration of content and initial training of online 
instructors, 

• System architecture and reliability 

• Custom application and course development 

• Helpdesk support and services 

• Vendor experience and qualification 

• Research and development future initiatives 

The institution must consider the scalability of their program 
and the ability of a course management system to 
accommodate the needs of continued growth, to ensure 
the platform selected will be able to grow with the institution 
as enrollments in their program grows. Those selecting a 
course management system should review the system 
architecture and reliability to determine redundancy, 
average downtime, and back up servers available, If the 
institution will need assistance in course development, it 
would be appropriate to investigate the system's support 
and resources available, their cost, and options for 
including such support in contract negotiations. For large 


institutions with high enrollments, helpdesk support may be 
a necessity and the course management system should 
clearly outline the services available. It is equally critical for 
institutions to consider the financial viability of a company 
that offers course management systems, and to review the 
vendor qualification and experience in the field. 
Examining these topics will help reveal 1) which companies 
are likely to survive the volatile technology market, 2) which 
companies invest in the future through research and 
development, and 3) which companies maintain a 
reputable product to service the needs of institutions in 
higher education. 

Evaluate RFP Responses 

RFP responses were evaluated by a committee selected to 
include faculty from a range of disciplines, online 
administrators, financial administrators, support staff, 
instructional technology personnel, and students, The 
evaluation focused on key dimensions relevant to the 
development and implementation of an online course: 1) 
Overall Platform Evaluation, 2) Intuitiveness/Ease of Use, 3) 
Navigation, 4) Reliability/Performance, 5) Course 
Communication Tools, 6) Course Assessment Tools, 7) 
Gradebook/Feedback Tools, 8) Content Creation, 9) 
Student Support Services, 10) Faculty Support Services, 11) 
Administrative Features/Reporting, 12) Other Requirements, 
and 13) Overall System Opinion. Each dimension was rated 
from 1 (did not meet expectations) to 5 (exceeds 
expectations). The dimensions and complete rating criteria 
are found in Appendix B. 

Conduct Campus Visits 

Once the top 3 to 5 finalists have been selected, you will 
want to invite each company for a campus visit to 
demonstrate the capabilities of their course management 
system. The goal of the campus visit is to examine ease of 
use, usability limitations, and intuitive functioning. The 
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campus visit gives evaluators the advantage of comparing 
usability issues when the system is being operated by an 
expert user who is familiar with the capabilities, functions 
and features of that particular course management 
system, The campus visit entailed the following: 

• Vendor Presentation: The vendor presentation 
allowed each vendor to display their strengths and 
provide an executive summary. 

• Guided Demonstration: To ensure a fair comparison 
between course management systems, it is important 
to provide explicit directions for the guided 
demonstration. Our guided demonstration of course 
functions and features consisted of a live 
demonstration that provided evaluation and 
comparison of the following course authoring 
capabilities: 1) Course Setup, 2) Lecture Creation, 3) 
Multimedia Upload 4) Groups, 5) Threaded Discussion, 
6) Tests, 7) Gradebook, 8) Chat/Whiteboard, and 9) 
Administrative tasks. A disk, with necessary content, was 
provided to each course management vendor upon 
arrival at the university, A university laptop computer 
containing no additional programs or plug-ins was 
used for the demonstration. Appendix C provides a 
complete description of the guided demonstration 
directions. 

• Hands-On Session: Conducted in a live course for 
faculty, administrator, and student groups, the hands- 
on session also provided an outlet for specific user 
groups to test the system and provide feedback about 
the experience and the usability/functionality of each 
system. 

• Question and Answer Session: Any remaining questions 
or concerns were addressed in the Q&A session 

.• Conclusion: The campus visit concluded with the 


vendor wrap up of their product. 

Each requested demonstration component was 
evaluated as not shown, minimal functionality, acceptable 
functionality or superior functionality. 

Select Finalist and Complete Negotiations 

The institution created one point of contact for vendors to 
direct all questions or concerns. Questions and answers 
were distributed to all vendors to ensure all vendors had the 
same information, and a vendor conference call was held 
to address questions/issues vendors had prior to submitting 
their RFP response. The RFP committee evaluated the 
proposals and completed a vendor proposal comparison 
matrix, to create a quantitative comparison of features 
based on the RFP matrix, 

All vendors completed a self-evaluation based on the 
matrix provided in the RFR which identified specific features 
and functionality the institution was seeking, Members of 
the student user group, administrator user group, and 
faculty user group all completed evaluation forms to 
record their feedback. The RFP committee completed a 
guided demonstration evaluation and an overall vendor 
evaluation form. All feedback and evaluation results were 
combined, tallied, and evaluated by the RFP committee. 
After the results were tabulated, the RFP committee made 
a unanimous decision and drafted a recommendation 
that was sent to the university's executive staff, Negotiations 
with the selected course management provider were 
completed by the executive staff. 

Create Implementation and User Education Plan 

Once a new online courseware provider is selected, 
migration of course content from the previous platform 
can start immediately if this issue has been covered in the 
RFP process. The courseware platform provider takes the 
lead on migration, gathering background information on 
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each course from the home institution and constructing a 
migration plan with a target "live" date. As the institution's 
previous platform provider has likely archived several 
terms/semesters worth of content, negotiations surrounding 
the original contract often need to take place between the 
institution and the previous platform provider to determine 
how much of the content can be transferred and how this 
transfer will occur, These negotiations should specify the IT 
configurations needed to transfer the content, and 
associated costs, as well as the extent of administrative 
support provided by the institution for the copying and 
saving of content. It is during this period of transfer from the 
old to the new platform that the home institution must 
create copies of all content that it would possibly need in 
the future. Cases of grade appeals, online classroom 
conduct issues, etc., should be reviewed to ensure that all 
relevant documentation is copied from the platform. 

The impending migration of content can provide ideal 
circumstances for the home institution to critically evaluate 
the quality of its existing online curricula. Depending on the 
online curriculum model used by the institution, the content 
prior to migration may have been housed in individual 
instructor's course shells, with a new version of the course 
essentially being offered each term, or the content may 
have been in a singular "master" version duplicated from 
each term to ensure consistency. Frequently, the former 
model is the one in place, and the home institution is faced 
with reconstructing a "master" version of each course that 
can be migrated to the new platform, as most online 
courseware providers will not migrate years and years' 
worth of online course iterations. A point of contact, a 
subject-matter expert, must be established for each online 
course so that the new platform's migration team can 
troubleshoot any content copying issues with informed 
individuals. As online operations at some institutions take 
place in spheres removed from face-to-face operations, 


the evaluation of existing online content can provide an 
opportunity to establish unified academic oversight 
procedures. 

Because the online course content must be critically 
reviewed in preparation for migration, it is essential that the 
implementation plan include ample time for content 
matter experts, including experienced online instructors 
and academic department personnel, to determine the 
action steps needed to create high quality "masters" for 
each course offered online, Many institutions conclude 
that large-scale course re-development needs to take 
place prior to migration, a process which involves the 
selection and training of new course developers. If the 
time frames allow, this re-development is most 
conveniently undertaken pre-migration, with the new 
online courseware platform providing new course shells for 
the developers to use and be trained within. However, it 
may not be possible to re-develop content prior to 
migration, meaning that the institution must be willing to 
suspend curricular quality improvements until after the 
platform has been implemented and users become 
comfortable with it, Either way, the shift to "master" versions 
of courses, versions that are duplicated from each term for 
the instructor to customize, is often a difficult transition for 
institutions whose instructors are used to having total 
freedom over the content in their courses. 

Implementing a new online courseware platform 
represents a cultural shift just as much as a technological 
one. For this reason, the user education plan created must 
be comprehensive and take a broad view of change 
management, a view that includes users' affective 
reactions to the new technology in addition to considering 
the technical skills that must be taught. The shift from 
instructor copies of courses to singular "master" copies 
represents one area of attention in change management 
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and is perhaps most effectively dealt with in terms of the 
institution's goals for academic quality and continued 
accreditation. Rationales for the decision to shift platform 
providers, and make changes to curricular structures, must 
be clearly and repeatedly transmitted via an inclusive 
communications plan. Involving upper-level administrators 
in academic affairs, as well as academic departmental 
representatives, in communicating with instructors is key to 
affirming that the decision to change platforms is about the 
shared values effective teaching and learning. 

Prior to or coinciding with the migration of content, it is 
essential to launch a wide-spread communications 
campaign, diverse in method and voice, to build 
excitement surrounding the technology implementation. 
Considering the target audiences for communication will 
determine the best methods, the overarching goal being 
to provide ample opportunities for stakeholders instructors, 
students, and administrators to ask questions, air concerns, 
and make suggestions regarding training, One model for 
a communications plan would combine asynchronous 
communications (e-mail and discussion boards) with 
synchronous opportunities (live chats and conference 
calls). Because it is difficult to ask questions about a 
platform the user has not experienced, the 
communications plan should also include an online 
training module or "guided tour" through the look and feel, 
features and functionality, of the new platform. During the 
RFP review or as part of contract negotiations, the home 
institution should request a demonstration course (the 
same one used during the campus visit or one modified to 
include the institution's branding) that can be used to 
acquaint users with the platform. As this demonstration 
course should be made available several semesters in 
advance of the platform implementation date, it is 
perhaps most helpful to have a demonstration course that 
simply provides an overview of the platform and does not 


require the user to begin training, recognizing the short 
shelf-life of training that is not immediately applied. 

The centerpiece of the user education plan will be the 
mechanisms used to train individuals to actually use the 
platform, and ideally, this training will begin within a month 
of the platform’s "live" date (of course, the administrators 
within the institution's online program will have been trained 
prior to this date), Recommended requests to the new 
platform provider include a student orientation course and 
an instructor training course, each of which can be 
customized by the home institution to include course 
policies and procedures. Detailed specifications for what 
these courses will accomplish (learning objectives) and 
include (text, graphics, interactive multimedia) should be 
clarified prior to delivery to avoid the possibility of extra 
costs for revision time. Again depending on the model of 
the institution, a separate training course may be needed 
for course developers, a course different in content from 
the instructor one, with an emphasis on course authoring, 
content locking (if applicable), and the terms and 
conditions of course development contracts. Courseware 
platforms often provide technical support different in kind 
and scope for course developers, and this support should 
be discussed during the RFP and contract negotiation 
period. 

Regardless of the particulars of the user education plan, 
the constant must be open communication. Training on a 
new technology in isolation, and without immediate 
opportunities to translate new knowledge into practice, 
often leads to poor performance and a lack of the user 
buy-in that is so critical for maintaining morale through a 
time of change. Accompanying each training resource 
(demonstration course, instructor and developer training 
courses) should be contact information for individuals at 
the home institution and from the new platform provider 
who can answer questions and provide "desk-side 
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coaching" for users who need it, 

Conclusion and Recommendations for Other 
Institutions 

Even the most comprehensive and detailed RFP process 
cannot ensure a positive experience and outcome. Many 
pitfalls and frustrations result from an over-reliance on the 
RFP process and a lack of attention to how that process will 
work at a given institution. Leaders of the RFP process must 
at all points be sensitive to the particulars of their institution's 
culture, political climate, and goals for quality and growth. 
With this in mind, the following recommendations are 
offered to guide an institution's execution of an RFP 
process: 

Communication and shared governance: 

When compiling an RFP review board, it is necessary to 
include both primary and what may be considered 
secondary stakeholders to ensure that a diversity of 
perspectives is represented in the process. Primary 
stakeholders usually include institutional administrators, 
representatives from enrollment and student services, 
board members, and online faculty and students. 
Expanding the stakeholders will guarantee broader 
representation across the institution and thus a potentially 
more successful implementation process. Broad 
representation of faculty and students on an RFP board is 
often difficult to achieve but nonetheless absolutely critical 
to an effective review process, in addition to including 
faculty members who are open advocates of an 
institution's online learning program, it is also important to 
represent the new or resistant faculty member or 
academic departmental representative. The same is true 
in determining student representation. A "token" student 
cannot represent with any degree of validity what is ahead 
for the student user of a new platform, Include both 
experienced and new online students, even if this involves 


conferencing in distance students via phone, as well as a 
representative from the institution's student governmental 
organization. This last student representative is important 
as many online platform implementations involve not only 
the fully-online course but also the supplemental 
courseware used in face-to-face courses and to 
administer electronic student evaluations. In addition to 
securing the potential for a more supported 
implementation, broad representation on an RFP review 
board also protects against one vendor or one campus 
group setting the agenda for the process or dominating 
negotiations, The more individuals on the review board, 
the more intense the negotiations, so selecting an effective 
review board leader/mediator becomes even more 
important. 

Extra-platform features and considerations: 

Understandably, the focal point of negotiations with a 
vendor is the features and functionality of the product. 
However, extra-platform components are often the most 
important to the successful implementation of a new 
courseware. Build into the RFP and RFP review process 
ample opportunity to learn about the platform's technical 
services and support in the areas of content migration and 
user support (help desk, support for course developers, 
administrative reviewers, face-to-face students using the 
platform, etc,). Have a clear sense of your users' potential 
technical support needs so that, if necessary, support over 
the basic helpdesk package can be negotiated. Likewise, 
involve the IT department to determine if the institution's 
existing technology infrastructure can support the 
processes surrounding implementation including the 
uploading of student and faculty information as part of 
each term's scheduling and to decide if the vendor or the 
institution will host the servers on which the information is 
kept. During these negotiations, the possibility for single 
sign-on access to institutional resources (online enrollment. 
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library, and courseware resources) can also be 
determined. 

Another extra-platform component that is frequently 
overlooked is the track record of the vendor its business 
plan, financial viability, user ratings, pricing schedule, etc. 
Request this information (often packaged as an annual 
financial report for stockholders) as well as investigate other 
institutions using the platform. Perhaps the "bottom line" of 
these extra-platform considerations is a reminder not to 
focus solely on the bottom line when selecting a vendor. 
Focusing more on price than value can result in the 
selection of online courseware that only meets the 
immediate, not future, needs of the institution. 

Institutional needs: 

To ensure the most productive experience with vendors, it is 
important for the institution to make its unique needs clear. 
Too often, RFP boards overlook the necessity of educating 
vendors about the institution's culture and particular goals. 
Provide background information on the institution including 
its mission and vision, history, student and faculty 
demographics, and strategic planning goals so that 
vendors can customize their presentations and packages. 
This information will enable vendors to determine what 
other online processes they can support (i.e. institutional 
research and assessment), as well as whether or not their 
product will be able to meet the institution's goals for 
program growth, Overlooking platform scalability can 
mean stunting the growth of an online program, 
Overlooking the particulars of an institution's culture and 
demographics can mean a contentious relationship 
between platform provider and user. 

Perhaps the most important institutional need is in the area 
of user education. An institution must have accurate 
knowledge of the various user groups affected by the 
platform transition, specifically the technology skill level, 


aptitude, and comfort with change of its users. The time 
frames for migration and training that seem feasible to 
platform providers and to online learning administrators 
may be completely out of sync with the reality of the users. 
Surveying online instructors and students is a time- 
consuming but necessary step before implementation 
time frames are determined. Additionally, 
communication across the institution is critical to ensure 
that the implementation of the new platform does not 
coincide with other new technology adoptions or 
institutional initiatives also requiring significant time 
investments from the same groups of users. 
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AppendixA 

List of Features by Major Area 

Course Features 

• Course Flome/lntroduction Page 

• Course Announcements 

• Notification of New Content/Posts (from students or 
instructor) 

• Calendar (course-related events added by instructor 
and personal or custom events) 

• Web-based Course Email tool (for instructor & students 
w/ no use of email client required) 

• Threaded Discussion Area 
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• Chatroom 

• Whiteboarding 

w/Math/Science Symbols 
w/ Foreign Language Symbols 

• Unit-Based Design (ability to arrange content based on 
units containing the lecture, quiz, threads, readings, 
etc. for a given week) 

• Feature-Based Design (ability to arrange content 
based on features so that all exams are in one area, all 
lectures are in another area, etc.) 

• Student Journal (private for student view only) 

• Student Journal (viewable/gradable by instructor) 

• Course Content Search 

• Web References 

• Equation Editing tools for students 

• Course Map 

• Content Pages 

• Online Glossary 

• Ability to Setup student groups (with auto creation of 
Threads, Chatrooms, Email groups, etc,) 

• Can lessons/exams be assigned to a specific group? 

• All portions of course content/tools/authoring areas 
can be accessed in 2 clicks or fewer. 

• Single browser window needed (no need for multiple 
browsers to be open to access threads, gradebook) 

• Student submission of assignments (dropbox) w/ variety 
of extensions allowed 

Testing Features 

• WYSIWYG Editing for test question creation (ability to 

60 


Upload images, audio files, etc.) 

• Multiple Choice 

• True False 

• Matching 

• Many Multiple Choice 

• Short Answer 

• Fill-in-the-Blank 

• Essay 

• Password Protection 

• Auto-Grading (objective questions only) 

• Instructor Feedback by question 

• Instructor Feedback for overall exam 

• Test Question Pooling/Randomization 

• Exam Question Import Tool 

• Practice Exams 

• Ability to Grade content items directly in the 
Gradebook and to provide individualized feedback to 
students 

• Test Preview 

• Timed Tests 

• Browser Lock Down during testing 

Gradebook 

• Student tracking (page loads and time on page) 

• Time/Date stamp of gradable items 

• Ability to Create Custom Gradeable Items 

• Can gradebook be edited while course is being 
offered (addition/deletion of assignments, weighting) 

• All Content Items Show in Gradebook as gradable 
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items (threads, tests, journal, chat sessions, student 
document uploads) 

• Ability to weight grades 

• Final Grade Calculation (updated & available to 
student throughout course session) 

• Gradebook Data Export Tool 

Course Authoring Tools 

• WYSIWYG Editing Tools 

• Equation Editing tools for instructor authoring 

• Course Setup Wizard 

• Authoring mode navigation has same navigational 
layout as the course itself 

• Image/Document Upload Wizard 

• Instructors can revise/edit the course during the 
course's offering 

• Instructor can toggle to student view without logging 
out 

• Course Look and Feel Customization 

• Timed Release of Content/Units 

• Group Management Tools 

• Duplication of All Course Content from term to term 
(lectures, quizzes, thread topics, etc.) 

• Duplication of Specific Course Items from term to term 

• Ability to upload streaming media files directly into the 
course 

Other Services Offered 

• Online Evaluation Services 

• 24x7x365 Helpdesk Support 


• University-specific toll-free helpdesk phone number 

• Free Online Faculty System Orientation/Training courses 

• Free Student Orientation Course 

• Online, context-sensitive, user manual for students 

• Backend Integration of Student Enrollment Data 

• Backend Integration of Courses/Multiple Sections 

• Online, Interactive FacultyTutorial/Courses 

• Online, context-specific, user's manual for instructors 

Online Campus Administration 

• Automatic browser update notification 

• Course listings that display current courses & archival of 
past courses 

• Ability to create and edit terms/date settings 

• Ability to create new course offerings 

• Ability to make changes directly to online campus 
informational pages (WYSIWYG Editing) 

• Ability to review user logins (instructors and students) 
from the administrator's interface 

• Ability to view login and system usage of students and 
instructors from the administrative interface 

• Does the login page have a password/user id look up 
feature? 

• Ability to create multiple user roles based on permission 
to campus/term/course levels 

• Ability to modify student data from one interface and 
have change replicate through system 

• Supports Recent Browser Releases 

• Multiple Operating Systems Supported (PCs, Macs) 
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• University Owns Courses/Content 

Appendix 8 

Overall Platform Evaluation 

Intuitiveness/Ease of Use 

• The layout of the course was appealing 

• Course menu and feature layout was easy to 
understand 

• The location and flow of course management features 
was logical 

• Course features and tools were tightly integrated (i.e. 
threaded discussion, whiteboards, and content 
authoring tools open within course layout and use 
consistent navigation) 

Navigation 

• Required less than 3 clicks to access course features 
and functions 

• It was easy to tell where I was in the course 

• Tasks were performed with speed and accuracy 

Reliability/Performance 

• Pages loaded in a reasonable amount of time 

• The system was free of error messages & "bugs" 

• The system did not crash or freeze up when attempting 
to perform the functions 

Course Communication Tools 

• The course system contained effective asynchronous 
communication tools (threaded discussion, email) 

• The course system contained effective synchronous 
communication tools (chat, whiteboard) 


Course Assessment Tools 

• Allowed easy creation and formatting of new test 
questions 

• Allowed easy import of test bank questions 

• Included appropriate test setup and security features 
(proctor passwords, browser lockdown, preset access 
dates, timed exams, etc.) 

• Tests provided students with feedback and scoring 
information 

• Easy creation and insertion of question pools 

Gradebook/Feedback Tools 

• Gradebook setup easy to understand and efficient 

• The gradebook included student activity tracking 

• Grading of student work from the gradebook was 
included 

• Creation of create custom grading items and modify 
grading scales 

Content Creation 

• The system allowed "live" editing with visual tools (no 
html required) 

• The content editing tools allowed easy upload and 
inclusion of media files (images, video, audio, 
hyperlinks) 

• Content tools allowed easy building of tables, 
utilization of special symbols, and formatting tools 
(bulleted lists, font size/color, and 

Student Support Services 

• The system offered online orientation to students 

• Help and tips for students were offered throughout the 
system 
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Faculty Support Services 

• The system offered online training and documentation 
to students 

• Help and tips for faculty were offered throughout the 
system 

Administrative Features/Reporting 

• The system enabled courses to be set up and 
displayed according to college, school, and term 

• The system included helpful standard reports 

• The system allows easy and efficient creation of new 
terms 

• The system allows multiple roles or levels of access for 
different user types 

Other Requirements 

• List other requirements here as appropriate 

Overall System Opinion 

• Rate your overall impression of the system and the 
company's ability to meet the needs of the university's 
online learning programs 

Appendix C 

Guided Demonstration of Course Functions and Features 

The guided demonstration of course functions and 
features will consist of a live demonstration to provide 
evaluation and comparison of the following course 
authoring capabilities: 1) Course Setup, 2) Lecture 
Creation, 3) Multimedia Upload 4) Groups, 5) Threaded 
Discussion, 6) Tests, 7) Gradebook, 8) Chat/Whiteboard, 
and 9) Administrative tasks. We would like you to complete 
these activities after your general company/system 
presentation. A disk, with necessary content, will be 
provided to each vendor upon arrival at the university. A 


university laptop computer will be used for the 
demonstration, 

Parti: Course Setup 

Step 1: From the course homepage, open the course 
authoring environment/tool. 

Step 2: Create a weekly unit/module. 

Step 3. Create the following module items: 

a. Lecture 

b. Discussion 

c. Quiz 

Step 4: Set the unit start/end dates for the module. 

Step 5: Show your course calendar feature and post an 
exam reminder for students. 

Step 6: Demonstrate how your system allows students to 
track their progress in completing modules, assignments, 
and due dates throughout the course. 

Part II: Lecture Creation 

Step 1: Create a text lecture using "weekl ,txt" from the disk 
provided. 

Step 2: Paste the text from the document into your text 
editing tool. 

a. Bold the first line "Welcome to the online 
classroom!" 

b. Change the size and color of the font. 

Step 3: Upload the image located on the disk called 
"mackay hall.gif" into your system and then insert it into the 
text. 

a. Resize the image. 

b. Move the image to a different place in 
the text. 
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Step 4: Using your platform's equation editing tools, create 
the equation (located on the disk provided) called 
"equation.doc" and place in the lecture, 

Step 5: Insert a 3 row/3 column table with a maroon 3 pt. 
border and a gold Background. 

a. Demonstrate creation of a table within 
a table cell. 

b. Demonstrate adding a column and/or 
row to the existing table. 

c. Demonstrate edition of an existing table 
(i.e. resizing columns, changing cell 
properties, colors). 

Step 6: Go to http://www.whitehouse.gov and copy the 
header images and first paragraph of text, and paste it into 
the lecture. 

Step 7: Undo your last action. 

Step 8: Copy and paste the text from pc mission- 
vision.doc into the lecture. 

Step 9: Create a hyperlinkto an internet site. 

Step 10: Does your system have a "full-screen" editing 
option? If so, please demonstrate. 

Step 11: Does your system have a spell check? If so, 
please demonstrate. 

Step 12: Save the lecture. 

Step 13: Open up the "student view" of the lecture and click 
on the hyperlink that you opened. Then navigate back to 
the lecture. 

Step 14: Open up the lecture again in the authoring 
environment and edit some of the text and then show the 
changes in the "student view". 

Part III: Groups 


Step 1: Create two student groups in the course called 
Group A and Group B. Please show all of the features that 
are automatically associated with each group. 

Step 2: Does your system have the capability to create 
group-specific assignments, 

threaded discussions, tests? If so, please demonstrate how 
an instructor would assign such an item to a group. 

Part IV: Threaded Discussion 

Step 1: Create a new weekly discussion thread with at least 
two discussion topics, and post a starter thread from the 
instructor. 

Step 2: Show how your threads expand/collapse and the 
student/instructor response window, (for this part of the 
demonstration, please use a threaded discussion 
that is already created in your demonstration course) 

Step 3: Show how your threaded discussion system allows 
instructors to easily sort threads based on the thread topic, 
author, date posted, read/not read. 

Step 4: Demonstrate/discuss the following: 

a. Instructor edits/reordering of posts 

b. Can students delete their posts? 

c. Can instructors view and grade student 
posts from the course gradebook? 

d. Printing of posts 

e. Any other features of your threaded 
discussion system that you feel 
distinguish your system 

PartV: Multimedia 

Step 1: Upload the "elearningoverview.ppt" file and link it 
into the course. Access the slideshow. 

Step 2: Upload the "radiotune.mp3" file. Link it into the 
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course and play it. 

Part VI: Tests 

Step 1: Create a quiz and demonstrate the following 
capabilities: 

a. turn exam access on and off using dates 

b. setting options for practice quizzes 
(multiple access) and exams (one-time 
access) 

c. set exam time limits 

d. set up a proctor password 

e. auto-grading options for immediate 
feedback and options for displaying 
exam information in the course 
gradebook 

Step 2: Using the file "exam.doc", upload the questions 
into the testing system. 

Step 3: Create a multiple choice question where the 
student can select multiple answers 

(format the question as follows), 

Question: The university has campus centers in the 
following cities : 

a. Fort Bliss, TX (correct) 

b. Independence, MO (correct) 

c. Shawnee, KS (incorrect) 

d. Charleston, NC (corrects 

e. Barstow, CA (correct) 

Step 4: Create a fill in the blank question. 

Question: Kansas City is located in: 

Answer: Missouri or Kansas 


Step 5: Create an essay question of your choosing that 
incorporates an image/audio file/ video etc. 

Step 6: Demonstrate the ability to create a question pool. 
Step 7: Demonstrate the ability to insert questions from a 
question pool into an exam. 

Step 8: Demonstrate the ability to create test sections or 
pages. 

Please submit your test question upload format to us in 
advance. We will prepare a file with several questions in 
your format and have it ready for you. 

Part VII: Gradebook 

Step 1: Demonstrate gradebook set up (you may use a 
previously created gradebook tor this portion of the 
demo). Show any items that are automatically created 
(For example, if you create a discussion thread, will it 
automatically show up as a gradable item?) 

Step 2: Create a custom gradable item called: Extra 
Credit worth 10 points. 

Step 3: Demonstrate student tracking and activity reports, 

Step 4: Demonstrate how correcting a quiz grade will 
impact the final course grade calculation. 

Part VIII: Chat & Whiteboard 

Step 1: Demonstrate setup of a separate chatroom called 
"office hours". 

Step 2: Enter the chatroom and make some entries, then 
show us the archived transcript, 

Step3: Demonstrate the following whiteboard capabilities: 

a. document presentation 

b. abilitytoenterformattedtext 

c. free hand drawings 
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d. moth symbol tool sets 

e. foreign language tool sets 

f. chat/polling/voting feature 

G. Archiving of whiteboard session 

Part IX: Administrative Capabilities 

Step 1: Create a new student enrollment; enroll a guest 
lecturer or auditor role into the course. 

Step 2: Show the standard reports that administrators can 
access (enrollments, faculty/student activity, etc.) 


Step 3: Show us howto set up a term. 

Step 4: Show us an example of a campus with courses in 
different levels (i.e. graduate/undergraduate, school of 
business vs. school of education). 

Step 2: Show the standard reports that administrators can 
access (enrollments, faculty/student activity, etc.) 

Step 3: Show us howto set up a term. 

Step 4: Show us an example of a campus with courses in 
different levels (i.e. graduate/undergraduate, school of 
business vs. school of education). 


( \ 
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